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Population protocols have been introduced as a model of sensor networks consisting of very 
limited mobile agents with no control over their own movement: A collection of anonymous 
agents, modeled by finite automata, interact in pairs according to some rules. 

Predicates on the initial configurations that can be computed by such protocols have 
been characterized under several hypotheses. 

We discuss here whether and when the rules of interactions between agents can be seen 
as a game from game theory. We do so by discussing several basic protocols. 

1 Introduction 

The computational power of networks of anonymous resource-limited mobile agents has been 
investigated in several recent papers. 

In particular, Angluin et al. proposed in [1] a new model of distributed computations. In this 
model, called population protocols, finitely many finite-state agents interact in pairs chosen by 
an adversary. Each interaction has the effect of updating the state of the two agents according 
to a joint transition function. 

A protocol is said to (stably) compute a predicate on the initial states of the agents if, in 
any fair execution, after finitely many interactions, all agents reach a common output that 
corresponds to the value of the predicate. 

The model was originally proposed to model computations realized by sensor networks in 
which passive agents are carried along by other entities. The canonical example of [1] corresponds 
to sensors attached to a flock of birds and that must be programmed to check some global 
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properties, like determining whether more than 5% of the population has elevated temperature. 
Motivating scenarios also include models of the propagation of trust [8]. 

Much of the work so far on population protocols has concentrated on characterizing which 
predicates on the initial states can be computed in different variants of the model and under 
various assumptions. In particular, the predicates computable by the unrestricted population 
protocols from [1] have been characterized as being precisely the semi- linear predicates, that is 
to say those predicates on counts of input agents definable in first-order Presburger arithmetic 
|18j . Semilinearity was shown to be sufficient in [ij and necessary in [2]. 

Variants considered so far include restriction to one-way communications, restriction to par- 
ticular interaction graphs, to random interactions, with possibly various kind of failures of agents. 
Solutions to classical problems of distributed algorithmics have also been considered in this 
model. Refer to survey [3] for a complete discussion. 

The population protocol model shares many features with other models already considered 
in the literature. In particular, models of pairwise interactions have been used to study the 
propagation of diseases [12], or rumors [Tj- In chemistry the chemical master equation has been 
justified using (stochastic) pairwise interactions between the finitely many molecules present 
\16\ 111] . In that sense, the model of population protocols may be considered as fundamental in 
several fields of study. 

Pairwise interactions between finite-state agents are sometimes motivated by the study of 
the dynamics of particular two-player games from game theory. For example, paper [9] considers 
the dynamics of the so-called PAVLOV behaviour in the iterated prisoner lemma. Several results 
about the time of convergence of this particular dynamics towards the stable state can be found 
in [9], and [10], for rings, and complete graphs. 

The purpose of the following discussion is to better understand whether and when pairwise 
interactions, and hence population protocols, can be considered as the result of a game. We 
want to understand if restricting to rules that come from a (symmetric) game is a limitation, 
and in particular whether restricting to rules that can be termed PAVLOV in the spirit of [9] is a 
limitation. We do so by giving solutions to several basic problems using rules of interactions as- 
sociated to a symmetric game. As such protocols must also be symmetric, we are also discussing 
whether restricting to symmetric rules in population protocols is a limitation. 

In Section [H we briefly recall population protocols. In Section [3l we recall some basics 
from game theory. In Section HI we discuss how a game can be turned into a dynamics, and 
introduce the notion of Pavlovian population protocol. In Section[5]we prove that any symmetric 
deterministic 2-states population protocol is Pavlovian, and that the problem of computing the 
OR, AND, as well as the leader election and majority problem admit Pavlovian solutions. We 
then discuss our results in Section [6l 

2 Population Protocols 

A protocol is given by (Q,!,, 1,(0,5) with the following components. 2 is a finite set of states. 
Z is a finite set of input symbols, i : £ ^ 2 is the initial state mapping, and CO . Q ^ {0, 1} is 
the individual output function. 5 C is a joint transition relation that describes how pairs of 
agents can interact. Relation 5 is sometimes described by listing all possible interactions using 
the notation {q\,q2) il'iTqi)^ even the notation 171^2 ^^'1^2' {^iiQi-i^'ii^'i) ^ ^ (with the 
convention that (<7i,<?2) ill 1^2) when no rule is specified with (171,^2) in the left-hand side). 
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The protocol is termed deterministic if for all pairs (^1,^2) there is only one pair {q'^^q'^) with 
(^1,^2) — > {q'l^q'i)- that case, we write 5\{q\,q2) for the unique q\ and 52(^1,(72) for the unique 

Notice that, in general, rules can be non-symmetric: if (^1,^2) (^'1)^2)' it does not neces- 
sarily follow that {q2,qi) (q'l^q'i)- 

Computations of a protocol proceed in the following way. The computation takes place 
among n agents, where n>2. A configuration of the system can be described by a vector of all 
the agents' states. The state of each agent is an element of Q. Because agents with the same 
states are indistinguishable, each configuration can be summarized as an unordered multiset of 
states, and hence of elements of Q. 

Each agent is given initially some input value from £: Each agent's initial state is determined 
by applying l to its input value. This determines the initial configuration of the population. 

An execution of a protocol proceeds from the initial configuration by interactions between 
pairs of agents. Suppose that two agents in state qi and q2 meet and have an interaction. They 
can change into state q\ and q'j if (^1,^25^15^2) the transition relation 5. If C and C' are 
two configurations, we write C ^ C' if C' can be obtained from C by a single interaction of two 
agents: this means that C contains two states qi and q2 and C' is obtained by replacing qi and q2 
by q[ and q'j in C, where (<?i,^2;^'n^2) ^ ^- execution of the protocol is an infinite sequence 
of configurations Co,Ci,C2, ■ ■ • , where Co is an initial configuration and C,- —i- C,+i for all / > 0. 
An execution is fair if for all configurations C that appear infinitely often in the execution, if 
C ^ C' for some configuration C', then C' appears infinitely often in the execution. 

At any point during an execution, each agent's state determines its output at that time. If 
the agent is in state q, its output value is (o{q). The configuration output is (respectively 1) if 
all the individual outputs are (respectively 1). If the individual outputs are mixed Os and Is 
then the output of the configuration is undefined. 

Let p be a predicate over multisets of elements of £. Predicate p can be considered as 
a function whose range is {0,1} and whose domain is the collection of these multisets. The 
predicate is said to be computed by the protocol if, for every multiset /, and every fair execution 
that starts from the initial configuration corresponding to /, the output value of every agent 
eventually stabilizes to p{I)- 

The following was proved in [H [2] 

Theorem 1 ([HIS]). A predicate is computable in the population protocol model if and only if 
it is semilinear. 

Recall that semilinear sets are known to correspond to predicates on counts of input agents 
definable in first-order Presburger arithmetic |18j . 

3 Game Theory 

We now recall the simplest concepts from Game Theory. We focus on non-cooperative games, 
with complete information, in extensive form. 

The simplest game is made up of two players, called / and //, with a finite set of options, 
called pure strategies, Strat{I) and Strat{II). Denote by Aij (respectively: Bjj) the score for 
player / (resp. //) when / uses strategy / € Strat{I) and // uses strategy j G Strat{II). 

The scores are given hy nxm matrices A and B, where n and m are the cardinality of St rat (I) 
and Strat{U). The game is termed symmetric if A is the transpose of B: this implies that n = m. 
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and we can assume without loss of generality that Strut {I) = St rat {II). 

Example 1 (Prisoner's dilemma). The case where A and B are the following matrices 

with T > R > P > S and 2R> T + S, is called the prisoner's dilemma. We denote by C (for 
cooperation) the first pure strategy, and by D (for defection) the second pure strategy of each 
player. 

As the game is symmetric, matrix A and B can also be denoted by: 

Opponent 



Player 





C 


D 


c 


R 


S 


D 


T 


P 



A strategy x G Strat{I) is said to be a best response to strategy y G St rat {II), denoted by 
X G BR{y) if 

Av^y < Ax^y (1) 

for all strategies z G St rat {I). 

A pair {x,y) is a (pure) Nash equilibrium if ;t G BR{y) and y G BR{x). A pure Nash equilibrium 
does not always exist. 

In other words, two strategies {x,y) form a Nash equilibrium if in that state neither of the 
players has a unilateral interest to deviate from it. 

Example 2. On the example of the prisoner's dilemma, BR{y) = D for all y, and BR{x) = D for 
all X. So {D,D) is the unique Nash equilibrium, and it is pure. In it, each player has score P. 
The paradox is that if they had played (C,C) (cooperation) they would have had score R, that is 
more. The social optimum (C,C), is different from the equilibrium that is reached by rational 
players {D,D), since in any other state, each player fears that the adversary plays C. 

We will also introduce the following definition: Given some strategy x' G Strat{I), a strategy 
X G Strut {I) is said to be a best response to strategy y G Strut {II) among those different from x' , 
denoted by :t G BR^x'{y) if 

Az^y < Ax^y (2) 

for all strategy z G Strut {I), z i^x' . 

Of course, the role of // and / can be inverted in the previous definition. 

There are two main approaches to discussing dynamics of games. The first consists in 
repeating games. The second in using models from evolutionary game theory. Refer to [13^ [19] 
for a presentation of this latter approach. 

Repeating Games. Repeating k times a game, is equivalent to extending the space of choices 
into Strut{I)^ and Strut{II)^: player / (respectively //) chooses his or her action x{t) G Strut{I), 
(resp. y{t) G Strut{II)) at time f for f = 1,2, • • • Hence, this is equivalent to a two-player game 
with respectively and choices for players. 

To avoid confusion, we will call actions the choices x{t),y{t) of each player at a given time, 
and strategies the sequences X = x(l), • • • ,x{k) and Y =y{l), - ■ ■ ,y{k), that is to say the strategies 
for the global game. 
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If the game is repeated an infinite number of times, a strategy becomes a function from 
integers to the set of actions, and the game is still equivalent to a two-player gam^. 

Behaviours. In practice, player / (respectively //) has to solve the following problem at each 
time t: given the history of the game up to now, that is to say 

Xt^i=x{l),--- ,;c(f-l) 

and 

Yt^i=y{l),--- ,y{t-l) 

what should I play at time t? In other words, how to choose x(t) G Strat[I)l (resp. y{t) G 
Strat{n)l) 

Is is natural to suppose that this is given by some behaviour rules: 

x{t)=f{x,^uyt-i), 

y{t) = g{X,^uyt-i) 

for some particular functions / and g. 

The Specific Case of the Prisoner's Lemma. The question of the best behaviour rule to 
use for the prisoner lemma gave birth to an important literature. In particular, after the book 
[1] , that describes the results of tournaments of behaviour rules for the iterated prisoner lemma, 
and that argues that there exists a best behaviour rule called TIT —FOR — TAT . This consists in 
cooperating at the first step, and then do the same thing as the adversary at subsequent times. 

A lot of other behaviours, most of them with very picturesque names have been proposed 
and studied: see for example [1], [5], [15] . 

Among possible behaviours is PAVLOV: in the iterated prisoner lemma, a player cooperates 
if and only if both players opted for the same alternative in the previous move. This name 
|14l [T7t Hj stems from the fact that this strategy embodies an almost reflex-like response to the 
payoff: it repeats its former move if it was rewarded by /? or T points, but switches behaviour if 
it was punished by receiving only P or S points. Refer to [17] for some study of this strategy in 
the spirit of Axelrod's tournaments. 

The PAVLOV behaviour can also be termed WIN-STAY, LOSE-SHIFT as if the play on the 
previous round resulted in a success, then the agent plays the same strategy on the next round. 
Alternatively, if the play resulted in a failure the agent switches to another action [17^ S] . 

Going From 2 Players to A'^ Players. PM^LOV behaviour is Markovian: a behaviour / is 
Markovian, if f{Xt^i,Yt^i) depends only on x{t — 1) and y{t — 1). 

Prom such a behaviour, it is easy to obtain a distributed dynamic. For example, let's follow 
[9], for the prisoner's dilemma. 

Suppose that we have a connected graph G = (y,E), with A'^ vertices. The vertices correspond 
to players. An instantaneous configuration of the system is given by an element of {C,D}^, that 
is to say by the state C or D of each vertex. Hence, there are 2^ configurations. 



^but whose matrices are infinite. 
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At each time t, one chooses randomly and uniformly one edge of the graph. At this 
moment, players / and j play the prisoner dilemma with the PAVLOV behaviour. It is easy to 
see that this corresponds to executing the following rules: 



cc - 


^ CC 


CD - 


^ DD 


DC - 


^ DD 


DD - 


^ CC. 



(3) 



What is the final state reached by the system? The underlying model is a very large Markov 
chain with 2^ states. The state E* = {C}^ is absorbing. If the graph G does not have any isolated 
vertex, this is the unique absorbing state, and there exists a sequence of transformations that 
transforms any state E into this state E*. As a consequence, from well-known classical results 
in Markov chain theory, whatever the initial configuration is, with probability 1, the system will 
eventually be in state E* [6]. The system is self-stabilizing. 

Several results about the time of convergence towards this stable state can be found in [9], 
and [To], for rings, and complete graphs. 

What is interesting in this example is that it shows how to go from a game, and a behaviour 
to a distributed dynamics on a graph, and in particular to a population protocol when the graph 
is the complete graph. 

4 From Games To Population Protocols 

In the spirit of the previous discussion, to any symmetric game, we can associate a population 
protocol as follows. 

Definition 1 (Associating a Protocol to a Game). Assume a symmetric two-player game is 
given. Let A be some threshold. 

The protocol associated to the game is a population protocol whose set of states is Q, where 
Q = Strat{I) = Strat{U) is the set of strategies of the game, and whose transition rules 5 are given 
as follows: 

('71,^2,^1,^2) S 5 

where 

• q\ = q\ when Mq^ q^ > A 

• q\ eBR^q^{q2) when Mq^^q, <A 
and 

• ^2 — ^2 when Mq^ qj > A 

• q2eBR^q,{qi) when Mq,,q^ <A, 
where M is the matrix of the game. 

Definition 2 (Pavlovian Population Protocol). A population protocol is Pavlovian if it can be 
obtained from a game as above. 

Remark 1. Clearly a Pavlovian population protocol must be symmetric.' indeed, whenever 
(^1,^2,^1,^2) S ^' one has (^2,^1 ,^2, ?'i) ^ ^- 



O. Bournez, J. Chalopin, J. Cohen, X. Koegler 



9 



5 Some Specific Pavlovian Protocols 

We now discuss whether assuming protocols Pavlovian is a restriction. 
We start by an easy consideration. 

Theorem 2. Any symmetric deterministic 2-states population protocol is Pavlovian. 

Proof. Consider a deterministic symmetric 2-states population protocol. Note Q = {+,— } its 
set of states. Its transition function can be written as follows: 



++ - 




+- - 




-+ - 






-y a a 



(4) 



for some a++ ,«+_,«_+,« 

This corresponds to the symmetric game given by the following pay-off matrix M 

Opponent 



Player 



taking threshold A = 1, where for all qi,q2 G }) 




-'9192 



-'9192 



2 if aq^q2 — ^1) 

otherwise. 



□ 



Unfortunately, not all rules correspond to a game. 
Proposition 1. Some symmetric population protocols are not Pavlovian. 



Proof. Consider for example a deterministic 3-states population protocol with set of states Q = 
{qo,qi,q2} and a joint transition function 5 such that 5i{qQ,qo) =qi, 5i{qi,qo) =q2 , 5i{q2,qo) = 

qo- 

Assume by contradiction that there exists a 2-player game corresponding to this 3-states pop- 
ulation protocol. Consider its payoff matrix M. Let M{qQ,qo) = j5o, M{q\,qQ) = , M{q2,qo) = Pi- 
We must have jSq > A,j3i > A since all agents that interact with an agent in state must change 
their state. Now, since qo changes to qi, q\ must be a strictly better response to than 172: 
hence, we must have pi > p2- In a similar way, since q\ changes to q2, we must have p2> Po , and 
since q2 changes to ^Oi we must have [5q> p\. From j8i > j32 > po we reach a contradiction. □ 



This indeed motivates the following study, where we discuss which problems admit a Pavlo- 
vian solution. 
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5.1 Basic Protocols 

Proposition 2. There is a Pavlovian protocol that computes the logical OR (resp. AND) of 
input hits. 

Proof. Consider the following protocol to compute OR, 



01 - 


11 


10 - 


11 


00 - 


00 


11 - 


11 



(5) 



and the following protocol to compute AND, 



' 01 - 


00 


10 - 


-> 00 


' 00 - 


00 


11 - 


-* 11 



(6) 



Since they are both deterministic 2-states population protocols, they are Pavlovian. 

□ 

Remark 2. Notice that OR (respectively AND) protocol corresponds to the predicates on counts 
of input agents jiq > 1 (resp. ni =0) where jiq, ni are the number of input agents in state and 
1 respectively. 

Remark 3. All previous protocols are "naturally broadcasting" i.e., eventually all agents agree on 
some (the correct) value. With previous definitions (which are the classical ones for population 
protocols), the following protocol does not compute the XOR or input bits, or equivalently does 
not compute predicate n\ = \ {mod 2). 



' 01 - 


01 


10 - 


10 


' 00 - 


00 


11 - 


00 



(7) 



Indeed, the answer is not eventually known by all the agents. It computes the XOR in a 
weaker form i.e., eventually, all agents will be in state 0, if the XOR of input bits is 0, or 
eventually only one agent will be in state 1, if the XOR of input bits is 1. 

5.2 Leader Election 

The classical solution [1] to the leader election problem (starting from a configuration with > 1 
leaders, eventually exactly one leader survives) is the following: 



LL - 


^ LN 


LN - 


^ LN 


NL - 


^ NL 


NN - 


^ NN 



(8) 



Unfortunately, this protocol is non-symmetric, and hence non-Pavlovian. 



O. Bournez, J. Chalopin, J. Cohen, X. Koegler 



11 



Remark 4. Actually, the problem is with the first rule, since one wants two leaders to become 
only one. If the two leaders are identical, this is clearly problematic with symmetric rules. 

However, the leader election problem can actually be solved by a Pavlovian protocol, at the 
price of a less trivial protocol. 

Proposition 3. The following Pavlovian protocol solves the leader election problem, as soon as 
the population is of size > 3. 



r L1L2 - 


-y LiN 


LiN - 


NL2 


L2N - 


-y NLi 


NN - 


-y NN 


L2U - 


-y NLi 


NLi - 


-y L2N 


NL2 - 


-y LiN 


LiU - 


L2L2 


. L2L2 - 





(9) 



Proof. Indeed, starting from a configuration containing not only A'^s, eventually after some time 
configurations will have exactly one leader, that is one agent in state L\ or L2. 

Indeed, the first rule and the fifth rule decrease strictly the number of leaders whenever there 
are more than two leaders. Now the other rules, preserve the number of leaders, and are made 
such that an L\ can always be transformed into an L2 and vice-versa, and hence are made such 
that a configuration where first or fifth rule applies can always be reached whenever there are 
more than two leaders. The fact that it solves the leader election problem then follows from the 
hypothesis of fairness in the definition of computations. 

This is a Pavlovian protocol, since it corresponds to the following payoff matrix, with thresh- 
old A = 4 

Opponent 





Li 


L2 


N 


Li 


1 


4 


1 


Player . 


3 


1 


1 


N 


2 


1 


4 



□ 

5.3 Majority 

Proposition 4. The majority problem (given some population ofOs and Is, determine whether 
there are more Os than Is) can be solved by a Pavlovian population protocol. 

If one prefers, the predicate nQ>ni on counts of input agents can be computed by a Pavlovian 
population protocol. 
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Proof. We claim that the following protocol outputs 1 if there are more Os than Is in the initial 
configuration and otherwise, 



' NY - 


YY 


YN - 


YY 


NO - 


YO 


ON - 


OY 


71 - 


Nl 


\Y - 


-> lA^ 


01 - 


NY 


. 10 - 


YN 



(10) 



taking 

• £ = {0,1},G = {0,1,F,A^}, 

• (o{0) = co{Y) = 1, 

• (o{\) = (o{N)=0. 

In this protocol, the states Y and A'^ are "neutral" elements for our predicate but they should 
be understood as Yes and No. They are the "answers" to the question: are there more Os than 
Is. 

This protocol is made such that the number of Os and Is is preserved except when a meets 
a 1. In that latter case, the two agents are deleted and transformed into a Y and a A'^. 

If there are initially strictly more Os than Is, from the fairness condition, each 1 will be 
paired with a and at some point no 1 will left. By fairness and since there is still at least a 
0, a configuration containing only and Ys will be reached. Since in such a configuration, no 
rule can modify the state of any agent, and since the output is defined and equals to 1 in such 
a configuration, the protocol is correct in this case 

By symmetry, one can show that the protocol outputs if there are initially strictly more Is 
than Os. 

Suppose now that initially, there are exactly the same number of Os and Is. By fairness, 
there exists a step when no more agents in the state or 1 left. Note that at the moment where 
the last is matched with the last 1, a 7 is created. Since this Y can be "broadcast" over the 
A'^s, in the final configuration all agents are in the state Y and thus the output is correct. 

This protocol is Pavlovian, since it corresponds to the following payoff matrix with thresh- 
old 2. 

Opponent 
N Y 1 
N i i 3~ 

Player Y 2 3 3 1 
2 2 2 1 
12 2 12 



□ 
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6 Discussions 

We proved that predicates on counts of input agents n>0, n = 0, n> m, where n,m are some 
counts of input agents, can be computed by some Pavlovian population protocols. 

It is clear that the subset of the predicates computable by Pavlovian population protocols 
is closed by negation: just switch the value of the individual output function of a protocol 
computing a predicate to get a protocol computing its negation. 

However, some work remains to be done to fully characterize which predicates can be com- 
puted by a Pavlovian population protocol. The first steps would be to understand the following 
questions. 

Question 1. Is mod 2, or equivalently the predicate n = \ [mod 2), computable by a Pavlovian 
population protocol? 

Question 2. Is > k, or equivalently the predicate n>k, for fixed k, computable by a Pavlovian 
population protocol? 

Notice that, unlike what happens for general population protocols, composing Pavlovian 
population protocols into a Pavlovian population protocol is not easy. It is not clear whether 
Pavlovian computable predicates are closed by conjunctions: classical constructions for general 
population protocols can not be used directly. 

As we said, Pavlovian Population protocols are symmetric. We however know that assuming 
population protocols symmetric is not a restriction. 

Proposition 5. Any population protocol can be simulated by a symmetric population protocol, 
as soon as the population is of size > 3. 

Before proving this proposition, we state the (immediate) main consequence. 

Corollary 1. A predicate is computable by a symmetric population protocol if and only if it is 
semilinear. 

Proof (of proposition): To a population protocol (2,£, l, ft), 5), with Q = {^i,--- associate 
population protocol (2U2',r, l,ftJ,5') with Q' = {q\,--- (0{q') = (0{q) for all q & Q, and for 
all rules 

qq aj3 

in 5, the following rules in 5': 



■ qq' - 




q'q - 


pa 


qq - 


- qW 


q'q' - 


qq 


qy - 


q'y 


q'y - 


qy 


yq - 


yq' 


. Yl' - 


yq 



for all 7 G QVJ Q' ,y ^ q,y ^ q' , and for all pairs of rules 



{ qr ap 
\ rq 5e 



14 



Playing With Population Protocols 



with q,r Q, the following rules in 5': 

qr' ajS 
r'q j8a 
V 5e 
q'r e5. 

The obtained population protocol is clearly symmetric. Now the first set of rules guarantees 
that a state in Q can always be converted to its primed version in Q' and vice- versa. By fairness, 
whenever a rule qq ajS (respectively qr aj3) can be applied, then the corresponding two 
first rules of the first set of rules (resp. of the second set of rules) can eventually be fired after 
possibly some conversions of states into their primed version or vice- versa. □ 
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